1. Field of the Invention
The present invention relates generally to the compression, cataloging and viewing of full motion videos and, more particularly, to the processing of compressed video data.
2. Description of Related Art
The infrastructure and process required to create and operate a video archive in the digital domain are well known in the broadcast video industry. The archiving process generally begins by digitizing and compressing the analog video using MPEG-1 or MPEG-2 compression, then moving the compressed video file to a long term storage. To preserve the contribution quality of the video, broadcasters generally select a high compressed bitrate (i.e., 15–40 Mbps), which allows the original video to be recovered with relatively high fidelity in spite of the lossiness of the MPEG compression scheme.
The high bitrate of the compressed video, however, presents considerable problems to the broadcaster's local area network and computer workstation infrastructure, when the video must be distributed for viewing and post-production work. The high network bandwidth and the amount of time required to transfer the assets throughout the plant places an upper limit on the number of concurrent transfers and severely constrains productivity. In response to this bandwidth problem, broadcasters create an additional copy of the video at a much lower compressed bitrate (i.e., 1.5–4 Mbps). This low bitrate file, referred to as a ‘proxy’ or ‘browse’ file, enables users to quickly download the video or to view it directly on computer monitors by utilizing a streaming video server. To facilitate the viewing of video assets outside the local area network, a second proxy file is often encoded at a very low bitrate (56–1000 Kbps), for streaming over low speed terrestrial lines.
After ingestion of the video, the next step in the archiving process is to create an entry for the video in the video library catalog. This entry contains metadata, which is information pertinent to the video. The contents and format of a video catalog record, normally broadcaster unique, facilitate the search and retrieval of video clips within the broadcaster's video library. Presently, there are commercially available video catalog applications (catalogers) that will automatically extract from an MPEG-1 or MPEG-2 video file metadata, such as closed caption text and the text of the actual audio program, obtained via speech recognition technology. Catalogers further extract metadata from the video by performing scene change analysis and creating a bitmap of the first frame after each cut or major scene transition. These bitmaps, referred to individually as a ‘thumbnail’ or collectively as a storyboard, are considered essential metadata because they enable the end user to determine very quickly the video content. Absent the storyboard, the end user is forced to view the video or, at a minimum, fast forward through a video to find the desired video segment. An additional feature of prior art catalogers is the capability to randomly access and play the proxy video file by double clicking on a storyboard thumbnail.
Further productivity gains can be achieved if the proxy file is a replica of the high-resolution video, where both files begin on the same video frame and have equal duration. When the browse file is a true proxy, a video production engineer is able to import several proxy files into a video editor and produce a program, creating an edit decision list (EDL). This EDL is subsequently exported to a high quality video editing suite that downloads the high-resolution version of the videos from the archive and executes the EDL to produce the air-ready material. Ideally, the broadcast editing suite retrieves from the broadcast server or archive only those segments of the high-resolution file that are specified in the EDL.
There are two timecodes associated with every video: an absolute and relative timecode. The absolute timecode is the SMPTE timecode recorded as the video is being shot. It usually reflects the actual time of day, but if the camera operator fails to properly set the SMPTE timecode generator on the camera, it may indicate any random clock time. Reporters and producers taking notes will record the SMPTE timecode while filming to enable them to quickly find important footage during post-production. It is for this reason that many archive librarians insist on preserving the absolute timecode as essential metadata when compressing and cataloging video.
The relative timecode is a timecode that is relative to the start of the video, and is often referred to as elapsed time. Many producers prefer to use relative timecode instead of absolute timecode during editing sessions, because it can simplify the arithmetic associated with calculating video clip duration. More importantly, it is more dependable than the absolute timecode.
The absolute timecode on a source video tape can be anomalous (e.g., missing, discontinuous, jump backwards in time, non-incrementing, non-drop frame mode, etc.). If a dub of the material is used as video source, it may not reflect the original timecode of acquisition. Moreover, regardless of whether longitudinal timecode (LTC) or vertical interval timecode (VITC) is being read as the source of timecode, tape dropout and other media defects can also result in corrupted SMPTE timecodes.
Accordingly, it would be advantageous to automatically verify and correct a bad timecode, when ingesting assets for permanent archival. It would be further advantageous for archive librarians to be able to re-stripe or change the timecode of the material, to suite their operational requirements. It would be yet further advantageous to be able to update the timecodes attached to each thumbnail in the storyboard, and any timecodes referenced in any existing description of the video and audio program, whenever the SMPTE timecodes in the high-resolution file have been modified.
However, in some instances, librarians may insist on preserving the original source timecode when archiving video. When video is later used to create new program material, the original source timecode facilitates the tracing of the content genealogy to the original source material. However, when the original source timecodes are maintained, the timecodes may contain gaps where shooting was halted and restarted. In the case of a compilation video, the video actually comprises a series of separate video clips.
In case of the noncontinuous absolute timecodes, EDL must be generated using relative timecode. It is for this reason that the relative timecodes of the proxy file must match the relative timecodes of the high-resolution file. This requirement in turn necessitates a method of synchronizing the relative timecodes of the proxy and high-resolution files.
Producing a high-resolution video and one or more proxy files with frame accurate synchronized relative timecodes of the proxy and high-resolution files is problematic, because two or more MPEG encoders and the source playout device must be started frame accurately, and the encoders must be capable of extracting SMPTE timecode (VITC) from the vertical blanking interval and storing the timecode in the MPEG Group of Pictures (GOP) header. (Some broadcasters may allow the encoders to encode alternately the locally produced house SMPTE timecode, instead.) Moreover, the encoders must not drop or repeat any frames during the encoding process, and the encoders must stop on the same video frame.
Although there are commercially available MPEG encoders that are capable of producing such proxy files, these encoders are prohibitively expensive and are not economical for an archive librarian planning to operate many ingest stations. Moreover, these high-end encoders often store the MPEG data in a vendor proprietary elementary stream format, which makes them uninteroperable with other MPEG decoders. Therefore, video files sent to another broadcast facility must be first remultiplexed into a MPEG compliant format. It is also undesirable from a business perspective to use a nonstandard storage format. Video quality and reliability are the normal criteria for selecting an encoder vendor. However, there is also a need to create proxy files using good-quality, but less capable MPEG encoders. An encoder that fails to store proper SMPTE time in the GOP header, for example, should not be eliminated from consideration, if it meets all other broadcaster requirements.
For all but a few high-end MPEG encoders, it is exceedingly difficult for an ingest application to consistently start multiple encoders and video sources with frame accurate timecodes. This can be attributed to the following factors: the inherent latency of MPEG encoders configured to encode at different rates and GOP configurations; operating systems, such as UNIX, AIX, and Windows NT, are not real-time operating systems with low interrupt service latency, consistent thread dispatch and a strict priority enforcement, packet delay when controlling encoders over a TCP/IP network; and inconsistent performance of audio/video equipment.
To facilitate the composition of EDL for the production of video programming, it is imperative that both the proxy and high-resolution video files be frame-accurately timestamped. Having to later re-encode proxy files to correct frame inaccuracies significantly increases the cost of operating of video archive. There is a demonstrable need to be able to adjust the timecodes of proxy files, when the starting frame of a proxy MPEG file is offset from the starting frame of its associated high-resolution MPEG file.
Presently, there are also problems with Storyboard Timecodes and EDL Composition. Conventional video catalogers capture either the elapsed time or the absolute timecode when creating thumbnails. This is problematic if the video source has anomalous timecodes, because the thumbnail timecode cannot be utilized for either EDL composition or play from offset commands. Storing both relative and absolute timecodes with each thumbnail would provide advantage over prior art. If discontinuous absolute timecodes were encountered, relative timecodes would be used instead, to create EDL and cue the MPEG player. Further advantage would be gained from modifying the EDL builder application to compose EDL statements using either relative or absolute timecode. Automatic switching to relative timecodes would allow librarians to maintain and view the original source timecodes, while generating a frame-accurate EDL.
Therefore, a need exists for the post-encoding modification of MPEG file timecodes, to create frame accurate timecodes. It is desirable to be able to adjust the timecodes of proxy files, when the starting frame of a proxy MPEG file is offset from the starting frame of its associated high-resolution MPEG file, to synchronize relative timecodes of the proxy and high-resolution files. It would be further advantageous to automatically verify and correct bad timecode when ingesting assets for permanent archival, and to be able to re-stripe or change the timecode of the material, to suite the operational requirements. It would be further advantageous to be able to update the timecodes attached to each thumbnail in the storyboard, and any timecodes referenced in any existing description of the video and audio program, whenever the SMPTE timecodes in the high-resolution file have been modified.